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Abstract 

Information about external world is delivered to the brain in the form of structured in time spike trains. 
During further processing in higher areas, information is subjected to a certain condensation process, which 
results in formation of abstract conceptual images of external world, apparently, represented as certain uniform 
spiking activity partially independent on the input spike trains details. Possible physical mechanism of 
condensation at the level of individual neuron was discussed recently. In a reverberating spiking neural 
network, due to this mechanism the dynamics should settle down to the same uniform/periodic activity in 
response to a set of various inputs. Since the same periodic activity may correspond to different input spike 
trains, we interpret this as possible candidate for information condensation mechanism in a network. Our 
purpose is to test this possibility in a network model consisting of five fully connected neurons, particularly, 
the influence of geometric size of the network, on its ability to condense information. Dynamics of 20 spiking 
neural networks of different geometric sizes are modelled by means of computer simulation. Each network was 
propelled into reverberating dynamics by applying various initial input spike trains. We run the dynamics 
until it becomes periodic. The Shannon's formula is used to calculate the amount of information in any 
input spike train and in any periodic state found. As a result, we obtain explicit estimate of the degree of 
information condensation in the networks, and conclude that it depends strongly on the net's geometric size. 



1 Introduction 

The ability of a biological object to obtain enough 
information about external world is essential for 
the object's survival. The information is deliv- 
ered to the central nervous system through vari- 
ous sensory pathways in the form of spike trains. 
The pathways' information throughput has been 
studyed for a long time in theoretical [22] and ex- 
perimental [23 research. The paradigm of those 
research is consistent with the assumption that the 



best situation is when the brain receives maximum 
information from sensory systems. But compare 
this with 10 . Another paradigm, which is as well 
mature, [23j, concentrates on self-organization of 
spike trains when primary sensory activity spreads 
to higher brain areas. Self-organization is ac- 
companied with information loss, [TS], Indeed, a 
kind of standartization of activity evoked by var- 
ious primary sensory inputs is obsereved exper- 
imentally in higher brain areas during olfactory 
[6] [38] and auditory [4] perception. In visual sys- 
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tem, simple examples are the transformation of 
scene representation to viewpoint-invariant, [7J or 
retinotopic-invariant ^32] coordinates. Here, spike 
trains in the optic nerve must depend on certain 
information (retinal position, viewpoint), which 
is removed in higher areas of conceptual repre- 
sentation. In the context of cognitive physiol- 
ogy, the process of reduction of information aimed 
at conceptual representation/recognition of exter- 
nal objects is known as information condensation, 
[21| . Usually pattern recognition phenomenon, 
which is closely related to information condensa- 
tion, is considered in parallel with training, see 
[HHZmi EH 130113111^, learning, P [TTl [SI HI] , 
or other plasticity, [16], in the corresponding net- 
work. In a biological network, the learning mecha- 
nism involves biosynthesis 27'. and is therefore very 
slow process, which requires seconds or minutes, 
[3]. At the same time, recognition of objects in vi- 
sual scene can be accomplished within 150 ms, or 
faster, citeThorpel, Thorpe. During this short pe- 
riod of time, the network has constant structure, 
but it is the spiking activity which evolves within 
it from information-rich at the sensory periphery 
into information-poor, representing concrete enti- 
ties/concepts at higher brain areas. 

We now put a question: What could be the 
physical mechanism of information condensation in 
a spiking neural system? One possible mechanism, 
[37| which operates at the level of single neuron, 
was discussed, see Fig. [T][2]and n. 2.1.1 12.1.11 be- 
low. In a reverberating spiking network, due to 
this mechanism the spiking dynamics should set- 
tle down to the same definite periodic activity in 
response to any stimulus from a definite set of var- 
ious inputs, and to another periodic activity in re- 
sponse to members of another set of inputs. If so, 
then the definite periodic dynamical state can be 
considered as an abstract representation of a fea- 
ture, which all stimuli from the definite set have in 
common. And this is just what is expected from 
the condensation of information. 

Our purpose in this work is to study how the 
ability of a simple spiking neural net to condense 
information in the above described sense depends 
on its physical parameter — the net's geomet- 
ric size. For this purpose we simulate dynamics 
of a net composed of five binding neurons placed 
equidistantly on a circle. The circle's radius, R, 
characterizes the net's geometric size. The net is 
fully connected, and propagation velocity is taken 
the same for all connections and all values of R. 
Thus, the variations in R are expressed exclusively 
in the variations in the interneuronal propagation 
times. Initially, the net is stimulated by a spike 
train of input impulses triggering each of five neu- 
rons at times {to, ti, . . . ,14}. Afterwards, the spik- 



ing dynamics is allowed to go freely until it settles 
down to a periodic one. This allows to figure out 
sets of input stimuli bringing about the same pe- 
riodic dynamics. The number of different periodic 
dynamics and the number of input stimuli in a set, 
which corresponds to a definite periodic dynamics, 
characterize the net's ability to condense informa- 
tion. We found that this ability depends strongly 
on the net's size. 

2 Methods 

2.1 The Binding Neuron Model 

The understanding of mechanisms of higher brain 
functions expects a continuous reduction from 
higher activities to lower ones, eventually, to ac- 
tivities in individual neurons, expressed in terms 
of membrane potentials and ionic currents. While 
this approach is correct scientifically and desirable 
for applications, the complete range of the reduc- 
tion is unavailable to a single researcher/engineer 
due to human brain limited capacity. In this con- 
nection, it would be helpful to abstract from the 
rules by which a neuron changes its membrane po- 
tentials to rules by which the input impulse sig- 
nals are processed into its output impulses. The 
coincidence detector, and temporal integrator are 
the examples of such an abstraction, see discus- 
sion by Konig et al., [20]. One more abstraction, 
the binding neuron (BN) model, is proposed as sig- 
nal processing unit, [34] which can operate as ei- 
ther coincidence detector, or temporal integrator, 
depending on quantitative characteristics of stim- 
ulation applied. This conforms with behavior of 
real neurons, see, e.g. work by Rudolph & Des- 
texhe, [26]. The BN model describes functioning 
of a neuron in terms of discret events, which are 
input and output impulses, and degree of tempo- 
ral coherence between the input events, see Fig. 
[1] Mathematically, this can be realized as follows. 
Each input impulse is stored in the BN for a fixed 
time, T. The r is similar to the tolerance inter- 
val discussed by MacKay, [23]. All input lines are 
excitatory and deliver identical impulses. The neu- 
ron fires an output impulse if the number of stored 
impulses, E, is equal or higher than the threshold 
value, A^o- In this model, inhibition is expressed in 
decreased r value. It is clear, that BN is triggered 
when a bunch of input impulses is received in a 
narrow temporal interval. In this case, the bunch 
could be considered as compound event, and the 
output impulse — as an abstract representation of 
this compound event. One could treat this mech- 
anism as binding of individual input events into 
a single output event, provided the input events 
are coherent in time. Such interpretation is sug- 
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Figure 1: Signal processing in the binding neuron model. [551 



gested by binding of features/events in largescale 
neuronal circuits, O (8] |9l |36] . 

ft would be interesting to characterize the BN 
input-output relations in the form of transfer func- 
tion, which allows exact calculation of output in 
terms of input. In our case, input is the se- 
quence of discrete arriving moments of standard 
impulses: Tin = {Ii,l2,l3,li, ■ ■ ■}■ The output is 
the sequence of discrete firing moments of BN: 
Tout = {/i, /2, . . . }. It is clear that Tout C T^. The 
transfer function in our case could be the function 
(t(Z), I £ Tin, which equals 1 if Z is the firing mo- 
ment, I G Tout, and otherwise. For BN with 
threshold A'^o the required function can be con- 
structed as follows. It is clear that the first A^'o — 1 
input impulses are unable to trigger neuron, there- 
fore a{h) = 0, . . . , (t(Z]Vo-i) = 0. The next input 
is able to trigger if and only if all A'^o inputs are 
coherent in time: 

(t(Z]Vo) = 1 if and only if Imq ^ h < t. 

In order to determine (T(ZjvQ+fc), fc > 1, one must 
take into acount all previous input moments, there- 
fore we use notation (TT;„ instead of a. The values 
•^f ""Tin (^iVo+fc) can be determined recursively: 

(^Ti^ilNo+k) = 1 if and only if iNo+k - h+i < r and 
crT„. [h) = for alH G {A; + 1, . . . , iVo + A; - 1}. 

The function (JTi^ describes completely the BN 
model for arbitrary threshold value No > 2 . 

2.1.1 Information Condensation in a 
Single Neuron 

It is worth noticing, that any firing (triggering) 
moment of a spiking neuron is determined by the 
moment of last input impulse, which just ensures 



that the triggering condition is satisfied. In a neu- 
ron, which needs more than one input impulse to 
fire, variations of temporal position of impulses, re- 
ceived just before the triggering one, do not infiu- 
ence the moment of emitting the output impulse, 
provided those variations are in resonable limits 
and arrival moment of the triggering impulse re- 
mains the same. 

r 

Ill I h 

^25 ^24 ^23 ^22 ^21 

— H 1— H 

^15 tl4 ti3 ti2 til 

Figure 2: Example of two different inputs into 
a single neuron, which produce identical out- 
puts. 

Thus, different input spike trains can produce ex- 
actly the same output. This looks like if some de- 
tailes of the input stimulus, which is composed of 
several impulses, were reduced/condensed in the 
output. In the BN, the triggering condition is 
that the number of impulses in the BN's inter- 
nal memory equals to A'^o. Consider BN with 
Ao = 4, which is stimulated with two different 
spike trains with input impulses arrival times Si = 
{ill, ii2, ii3, ii4, iis}, ill < ti2 < iis < tu < ti5, 

and 5*2 = {i21, i22, i23, i24, i25}, i21 < i22 < t23 < 

t2i < t25- Let the arrival moments satisfy the fol- 
lowing conditions: 

tl4 — til < r, t25 — t22 < T, 
t24 — ^21 > T, ti4 = t25 = to ■ 
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In this case, both Si and S2, if fed to the BN, will 
produce exactly the same output, namely, the sin- 
gle impulse at moment to- This is illustrated in 
Fig.H 

2.2 The Network Model 

As a reverberating spiking neural net we take the 
net of five neurons placed equidistantly at a circle 
of radius R, see Fig, [S] Each neuron has threshold 
A^o = 4, and internal memory, t — 10 ms. The 
net is fully connected. The connection lines are 
characterized with length and propagation veloc- 
ity, V, which is the same in all lines. For R fixed, 
there are two types of connection line, the short 
one, with propagation delay d, and the long one 
with propagation delay D. Each neuron has addi- 
tionally the external stimulus input line, which is 
used to start the net dynamics. Single impulse in 
the stimulus input line delivers to its target neuron 
just threshold excitation. This causes firing at the 
moment of the stimulus impulse arrival. For nu- 
merical simulations we use 20 networks of different 
sizes, see Table [1] The propagation velocity in any 
interconnection line is taken v = 0.1 m/s. 

2.2.1 Numerical Simulation 

As programming language we use Python under 
Linux operating system. The dynamics was mod- 
elled by advancing time with step dt = 200 fis. 
The delay values d and D, when measured in the 
dt units, were rounded to the nearest from below 
integers, see Table [1] As a result, the simulating 
program operates in whole numbers with no round- 
ing errors involved. 

Each single tick of the program, the network's 
tick, advances time by dt, and consists of three par- 
tial ticks, which are performed in the given order. 
Namely, (i) input tick, which advances time in the 
input lines, (ii) axonal tick, which advances time 
in the internal connection lines, (iii) neuronal tick, 
which advances time in the neurons. 

This manner of updating states can be treated 
as synchronous in a sense that each component of 
the network has the same physical time when the 
network's tick is complete. On the other hand, 
viewed as interneuronal communication process, 
the dynamics should be treated as asynchronous 
due to nonzero propagation delays. 

During the step (ii), a neuron can get impulse 
into its internal memory. If a neuron appears in the 
state "Fire" as a result of the network tick, then 
the output impulse it produces can appear in the 
connection lines only during the next network tick. 
This introduces effective delay of one dt between 
delivering the triggering impulse to a neuron and 
emitting output impulse by that neuron. 



2.3 Data Acquisition Algorithm 

2.3.1 Set of Stimuli 

The net was entrained to reverberating dynamics 
by applying initial input spike train of five im- 
pulses, one triggering impulse per neuron, at times 
(in dt units) {to = 1, ti, t2, ts, t4}. The triggering 
moment of neuron # is taken 1 for all stimuli 
in order to exclude rotational symmetry between 
the stimuli applied. Other four triggering moments 
run independently through the set {1,2,..., tmax}, 
where tmax is choosen proportional to R for each 
net size. In choosing tmax, we follow two different 
paradigms. In the first paradigm of short stim- 
uli we restrict the overall duration of the stim- 
ulus train with the value tmax ~ d. Thus, the 
set of stimuli has different stimuli. If ti < d, 
i = 0, . . . , 4, then any neuron in the net never 
obtains impulse from other neurons before it ob- 
tains its external input stimulation. In the second 
paradigm of extended stimuli, we restrict the over- 
all duration of the stimulus train with the value 
tmax ~ M, which is about three times longer than 
d for each network (see Table [T]). Here, all stim- 
uli {to = 1, t\,t2,t3,t4}, which were presented to a 
network, cover the set of different trains, which 
equals from 625 different stimuli for net #1 to 
100 000 000 different stimuli for net #20 (see Table 
[T]). The stimuli were sampled in accordance with 
standard algorithm of 4-digit counter. Namely, we 
started from stimulus {1,1,1,1,1}, the next stim- 
ulus is obtained by advancing ti by 1, and so on. 
The stimulus next to {1, M, 1, 1, 1} is {1, 1, 2, 1, 1}, 
the one next to {1, M, M, 1, 1} is {1, 1, 1, 2, 1} and 
so on, until stimulus {1, M, M, M, AI} is presented. 

In the extended paradigm, the late external in- 
put impulse can enter corresponding neuron after 
it received impulses from neurons already triggered 
by early external input impulses. 

The second paradigme is in concordance with 
visual information processing, [2j where activity 
from higher brain areas, which was invoked due to 
visual stimulation at earlier time, is retroinjected 
to areas VI and V2 in the primary visual cortex, 
where it interacts with activity invoked by visual 
input at later time during perception. 

2.3.2 Figuring out Periodic States 

After the last input impulse from the train {to — 
I,ti,t2,t3,t4,} reachs its target neuron, the pro- 
gram begins appending at each time step the in- 
stantaneous state of the net to a Python list. The 
instantaneous state consists of states of all 20 con- 
nection lines and all 5 neurons (see Fig. |4]). Before 
appending, the program checks if the current in- 
stantaneous state was already included in the list. 
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Figure 3: The network, used for simulations. Here {to,ti,t2,ts,t4} — is the input spike train, d, D — are the 
propagation delays in the connection lines. Any line can be either empty, or propagating one impulse. 



net # 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


R, mm 


0.029 


0.057 


0.086 


0.114 


0.143 


0.171 


0.200 


0.229 


0.257 


0.286 


d, dt 


1 


3 


5 


6 


8 


10 


11 


13 


15 


16 


D, dt 


2 


5 


8 


10 


13 


16 


19 


21 


24 


27 


M, dt 


5 


10 


15 


20 


25 


30 


35 


40 


45 


50 


net # 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


R, mm 


0.314 


0.343 


0.371 


0.400 


0.429 


0.457 


0.486 


0.514 


0.543 


0.571 


d, dt 


18 


20 


21 


23 


25 


26 


28 


30 


31 


33 


D, dt 


29 


32 


35 


38 


40 


43 


46 


48 


51 


54 


M, dt 


55 


60 


65 


70 


75 


80 


85 


90 


95 


100 



Table 1: Dimensions of networks used for simulations; dt = 200 iis. 
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11 
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49^ 


544 


58^ 


63^ 















Table 2: Distinct periods in dt units of found periodic states in the short stimuli paradigm. Superscript denotes 
the number of different periodic states with this period. 
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If it was, then the periodic dynamical state is found 
with its complete cyclic trajectory covered by in- 
stantaneous states between the inclusion and the 
last record in the list, inclusively. Measures are 
taken in order not to count the same cyclic trajec- 
tory, which was entrained at its different points, as 
different periodic states. 

The data for each net were stored in two 
MySQL tables. Table STATES included the se- 
rial number of any periodic state found, one ele- 
ment from the corresponding cyclic trajectory, and 
period of the state (see Fig. |4]). Single record 
in the INPUTS table included the input stimulus 
{to = I,ti,t2,t3,t4}, the serial number of the pe- 
riodic state it leads to (this number is for fading 
dynamics), and relaxation time, namely, the time, 
which is spent between the last external input im- 
pulse is delivered and the net's entrance moment 
into the periodic regime. 

3 Results 

3.1 Characterization of Periodic 
States Found 

After initial stimulation, networks from #1 to #7 
entrain to periodic activity after any stimulation, 
and networks #8 to #20 either entrain to peri- 
odic dynamics, or stops from any activity after 
some time. This takes place for both short and 
extended stimuli. The number of different periodic 
states found for each network is shown in Fig. (5] 
Here, the maximal number of periodic states ob- 
tained with short stimuli is 18, and this number is 
achieved for net numbers from 3 to 7. Exact values 
of periods, and number of different periodic states 
with this period is shown in Table [5] for short stim- 
uli. We omit similar table for extended stimuli. 
The maximal number of different periodic states 
obtained with extended stimuli is 485, which is 
achieved in the net number 9. The maximal num- 
ber of different periodic states with the same pe- 
riod is 294 for period 50-dt in net #9. It should be 
mentioned that two periodic states, which can be 
turned into eachother by suitable renumeration of 
neurons, were considered as different. 

It is evident that in the network of five neurons 
with threshold 4, each neuron is triggered the same 
number of times during period. Indeed, expect 
that neuron 714 fires k4 times, and any of the other 
four fires less during period: k4 > ki, i = 0, 1, 2, 3. 
In order to be triggered k4 times, must obtain 
not less than 4^4 input impulses during period. 
But it can obtain only fco-|-fci-|-fc2-|-fc3, which is less 
than required. Similarly, situation when ^4 = fca 



and ks > ki, i — 0,1,2 leads to contradiction, and 
so on. 

This number of triggering is either 1 or 2 for 
trajectories found, see examples in Fig. |6] Some 
nets have only one periodic state, which coresponds 
to synchronous firing of all 5 neurons and symmet- 
rical states of connection lines at any moment of 
time. This is the case for nets number 1 and from 
15 to 20 for short stimuli, and for nets number 19 
and 20 for extended stimuli. 

3.2 Condensation of Information 

In order to estimate the degree of information con- 
densation in the course of transformation of an ex- 
ternal spike train into a certain periodic state of the 
net, one needs to calculate information amount, 
which is delivered by specifying a spike train, and 
which is delivered by specifying the state, it leads 
to. 

3 2 0,1 4 
I 1 — I 1 — h state #319 

' ' ' '50 

2,3 0,4 1 
I 1 1 U state #107 

' ' '50 

0,3 1,2 4 0,3 1,2 4 

. 1 1 f 1 1 U state #199 

III I I igg 

Figure 6: Examples of periodic states found 
for net number 9 in the extended stimuli 
paradigm. Spikes indicate the firing moments, 
labels near each spike give numbers of neurons, 
firing at this moment. The two upper trains 
show states with period 10 ms, the lower one - 
with period 13.2 ms. 

This can be done by wellknown Shannon's formula 

m 

H = -^Pilog2Pi, (!) 

i 

where pi is the probability to obtain case num- 
ber i from a set of cases. At the input end we 
have the set of d*, or different external input 
spike trains. In our statement of the problem, it 
is natural to consider all external input trains as 
equally probable. If so, then information delivered 
by specifying certain train is 

Hs=Uog^d, (2) 

for the short stimuli paradigm, and 

i/e = 41og2M, (3) 

for the extended stimuli paradigm. 

While estimating information, delivered by 
specifying certain periodic state, one should take 
into account that probabilities of different periodic 
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mysql> select * from STATES_5_9 where nuiii=269; 

+ + 

+ + 

I num I state 

I period I 

+ + 

+ + 

I 269 I 1 1 8 8 17 17 24 15 15 24 8 8 8 17 17 8 
False False 
False False 
False False 
False False 

False True 49 1 
I 82 I 

+ + 

+ + 

1 row in set (0.02 sec) 

iiiysql> 

Figure 4: Example of single record in the MySQL table STATES. The first field (num) is numerical, and gives 
the serial number of periodic state found. The second field (state) is a string, which describes instantaneous 
state from the periodic state found (a point from the cyclic trajectory, which represents the whole trajectory, or 
periodic state). The first 20 numbers in the string describe states of all connection lines: '0' means that the line 
is empty, positive number specifies after how many ticks the propagating impulse will rich the targeted neuron. 
The next five chunks confined between the "next line" symbols describe states of neurons. The first number in 
each chunk is the "kick" — the total number of impulses obtained by neuron after the axonal tick was complete. 
During the neuronal tick, corresponding to that axonal tick, the "kick" is utilized and set to zero. The next 
boolean in the chunk indicates if the neuron is in the " Fire" state. The next boolean indicates if the neuron has 
any impulses in its internal memory. If it has, then next couples of numbers (up to three couples) describe those 
impulses. In this example, neuron #4 has 1 impulse with time to leave 49-dt. The third field (period) specifies 
period (in dt units) of this periodic state. 





Figure 5: Number of different periodic states found for each net with short (left panel), and extended (right 
panel) stimuli. 
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states are not the same. In order to calculate prob- 
ability p„ of a periodic state C'„, we calculate the 
number of input spike trains leading to the Cn, 
namely, T„, and divide this number by the total 
amount of different input stimuli (see histogram 
for T„ in Fig. [3 



Pn 



d4' 



for the short stimuli paradigm, and 



Pn 



(4) 



(5) 



for the extended stimuli paradigm. Then we use 
Eq. ([1]) with probabilities of individual periodic 
states found in accordance with ((SJ to cal- 
culate information which should be ascribed to 
any periodic state. In this calculations, we treat 
uniformly with others the external input stimuli, 
which lead to fading dynamics. Correspondingly, 
the state with no activity is treated uniformly with 
periodic states. This is in the contrast with data 
presented in Fig. [5l and Table [2l where the state 
with no activity is excluded. 




10000 20000 30000 40000 

size of conceptual domain 

Figure 7: Histogram of conceptual domain 
sizes for net #9 in extended stimuli paradigm. 
The bin size is 518. 23 domains with sizes from 
50764 to 193732 are not presented. 

As it could be expected, the information 
amount in an external input spike train increases 
as logarithm of the net size, in accordance with 
Eqs. ([2]), ((3|, varying from 25 to 81 bits for short 
paradigm, and from 37 to 106 bits for extended 
paradigm. Information, which could be ascribed 
to a periodic state, depends on the net size in a 
more complicated manner, see Fig. [S] A remark- 
able feature is a kind of plateau between net #3 
and #9 for both short, and extended stimulation 
paradigme. In the plateau, the 




Figure 8: Dependence of information amount, 
which is ascribed to periodic state, on the net 
size. Curve 'e' corresponds to extended stimuli 
paradigm, 's' — to the short stimuli one. 

information of a periodic state, calculated in accor- 
dance with Eq. (m with probabilities found due to 
Eqs. Q, ((5}, varies between 6.93 and 7.33 bits 
for extended stimuli, and between 3.17 and 3.46 
for short stimuli. In the plateau, the degree of in- 
formation condensation calculated as input infor- 
mation divided by the periodic state information, 
varies between 9.02 and 12.27 for extended stim- 
uli, and between 11.2 and 19.31 for short stimuli. 
Out of a plateau range, for greater net sizes the 
amount of information in a periodic state drops 
sharply due to simplification of the set of peri- 
odic states. Namely, the total number of periodic 
states decreases to one, with the second one with 
no activity, and the probability is distributed very 
unevenly between this two states. This leads to 
the degree of information condensation as high as 
41000 for extended stimuli, and 690 for short ones. 

4 Discussion 

Any reverberating spiking neural net can repre- 
sent complicated dynamical behavior. If the net's 
instantaneous states can be described with whole 
numbers, then the net will inevitably either en- 
train to periodic dynamics, or stop its activity at 
all. In this study, it is appeared that a very simple 
net of Fig. [3] can be engaged into a considerably 
large set of different periodic activities. It is not 
clear which part of all possible in this net periodic 
states was discovered in our simulation. As it fol- 
lows from comparison between short and extended 
stimuli paradigm, the number of different periodic 
states found increases with increasing range of in- 
put stimuli. Certainly, this increase must saturate 
somewhere. This is because any two different peri- 
odic regimes are represented by their cyclic trajec- 
tories, which has no common points (instantaneous 
states). On the other hand, the total number of 
instantaneous states the network can have is finite 
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due to finiteness of the set of states of each element 
the network is composed of. 

The number of periodic states found in a net 
depends on the net's geometric size. Variations 
in the net's size display themselves exclusively in 
variations of interneuronal propagation delays d 
and D. On the other hand, the duration of neu- 
ronal internal memory, r, is the same for net of any 
size. Thus, it is namely the relationships between 
the times an impulse spends for travelling between 
neurons, and time it is allowed to spend in a neu- 
ron waiting for additional impulses, which controls 
possible number of periodic states. 

It is worth noticing that each net has one com- 
pletely synchronized periodic state. The com- 
pletely synchronozed state is stable and achieved 
during finite time. This is in the contrast to the 
case of pulse-coupled oscillators with delayed exci- 
tatory coupling, (see, e.g. Ref. [IS]). 

In the four-dimensional set of stimuli we used, 
the neighbouring stimuli differ from eachother by 
one dt in one of four dimensions. This can be 
treated as analogous representation of some reality. 
The set of periodic states should be considered as 
a set of discrete entities due to qualitative differ- 
ence between any two states. This conforms with 
a paradigm discussed in cognitive physiology, [21] . 
The process of transformation of initial analogous 
inputs into a discret set of periodic states implies a 
loss of information and can be treated as conden- 
sation of information. 

If we take a set of input stimuli, any of which 
leads to the same periodic state, then that periodic 
state can be considered as an abstract/conceptual 
representation of a feature, which all stimuli from 
the set have in common, and the corresponding set 
could be named as "conceptual domain". What 
kind of feature or concept does the conceptual do- 
main represent? If our net was trained to recognize 
a certain real feature, then it would be that feature. 
In the context of this study, the common feature is 
that all stimuli from the conceptual domain engage 
namely this net into namely this periodic dynam- 




Figure 9: 2-diniensional cross-section of the 4- 
dimensional space of inputs for network ^9 in 



extended stimuli paradigm, left, cross-section 
is made with plane (^3 = 23, i4 = 23), right, 
(ts = 21,^4 = 23). Origin for (^1,^2) is in 
the upper left corner. Both ti and t2 run 
through the set {1,2,..., 45} of values. Num- 
bers in polygons indicate serial numbers of cor- 
responding periodic states. 

It would be interesting to have a look on 
the topology of the conceptual domains in the 4- 
dimensional space of all stimuli. For this pur- 
pose we figured out 2-dimensional cross-section 
of the input stimuli (see Fig. [9)1. In the cross- 
section, a typical conceptual domain is composed 
of several coherent clusters disconnected with ea- 
chother. The histogram of sizes of conceptual do- 
mains found is given in Fig. [7] 

Decomposition of the whole set of input stimuli 
into a number of conceptual domains represented 
by corresponding periodic dynamics resembles ap- 
proach in analyses of multivariate datasets, see re- 
view in [I3]. The difference is that here the 4- 
dimensional dataset (a conceptual domain) is rep- 
resented by unidimensional cyclic trajectory, which 
corresponds to the domain, and the trajectory is 
composed of points/vectors, which have other di- 
mension than the dataset vectors (see Fig. [4}. 
Nevertheless, having the network, the whole cyclic 
trajectory can be reproduced starting from its any 
single point. Thus, here the datasets are reduced 
down to individual points. This is in concordance 
with the information condensation idea. 

Why do we stick ourselves with namely the pe- 
riodic dynamics? The answer is related to the 
memory /learning problem, even if we do not con- 
sider any plasticity in this study. It is known [3] 
that modification of synaptic strength may hap- 
pen due to repetitive delivery of impulses to those 
synapses. Periodic dynamical states are just well 
suited for such repetitive delivery. All other dy- 
namical behaviors are of transient type, and have 
less chances to cause plastic changes in biological 
network. On the other hand, successful perception 
expects ability to report about what was perceived, 
which is impossible without memory. 

Unfortunately, we cannot draw this biological 
analogy too far. Real biological network includes 
not only excitatory, but about 4% of inhibitory 
neurons, [TS]. Representing this characteristic in 
a model network requires to have at least 25 neu- 
rons in it. This dramatically increases the number 
of possible external stimuli, which requires a quali- 
tative change in our approach as regards the speed 
of simulations and analysis of obtained data. 

Finally, what happens if we use another neu- 
ronal model in the network? Our opinion is that re- 
sults will be qualitatively similar. Using the bind- 
ing neuron here is natural, since it represents in 
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refined form what a spiking neuron does with sig- 
nals it receives. Additionally, the BN model easily 
allows to develop a program operating in whole 
numbers. This excludes possible dynamical arte- 
facts due to rounding errors. 

In future work, it would be interesting to com- 
pare results, if another spiking neuron model is 
used in the network, to study the topology of 
conceptual domains and how the topology could 
change if a plasticity is introduced in the network 
model. 

5 Conclusions 

A network composed of spiking neurons is able to 
condense information due to the fact that differ- 
ent initial stimuli could lead the network to the 
same periodic dynamics. This happens by means 
of initial/basic condensation of information in spik- 
ing neurons, as it is described in n. 2.1.1 I2.1.T1 
above. The network's geometric size, which deter- 
mines the interneuronal transmission delays, has 
considerable influence on the net's ability to con- 
dense information, mainly due to influence on the 
number of different periodic states the network can 
have, see Fig. [S] The latter has influence on the 
amount of information, which should be ascribed 
to a single periodic state, see Fig. |8l As a re- 
sult, the degree of information condensation varies 
between 9 and 41000, see n. 3. 33.21 for details. 

The networks considered here are too primi- 
tive to have reliable biological implications. At the 
same time, numerical parameters, see Table [1] and 
n. 2. 22.21 such as network sizes and spike propaga- 
tion velocity, are taken corresponding to biological 
data. The threshold value 4 does not contradict 
to biological reality, as experimentally registered 
thresholds are between 1 and 300. The BN inter- 
nal memory duration, r, is commensurable with 
halfdecay time of the excitatory postsynaptic po- 
tentials (EPSP). Thus, in the framework of this 
extremely simple model, one could expect that the 
ability of biological neural network to condense in- 
formation should depend on its geometric size, or 
on the relationships between interneuronal trans- 
mission delays and the EPSP halfdecay time. 
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